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Abstract 

We construct a basis of sparse eigenvectors for the TV- dimensional dis- 
crete Fourier transform. The sparsity differs from the optimal by at most 
a factor of four. When TV is a perfect square, the basis is orthogonal. 



1 Introduction 

The A-dimensional discrete Fourier transform (DFT) can be viewed as an 
N x N complex- valued matrix D. If we set w = e - 27Ti/N , then we define 
the entry in the j'-th row and fe-th column of D to be 



/VN 



(where j and k run from to N — 1). 

The DFT has a number of strange features. For instance, if we apply 
the DFT to a vector four times, we recover the original vector. In other 
words, 

D 4 = I. 

It follows that any eigenvalue of D must be a fourth root of unity, namely 
an element of {±1, ±i}. 

If two eigenvectors share the same eigenvalue, then any linear combina- 
tion of them is also an eigenvector (with the same eigenvalue). Therefore, 
there are four vector subspaces corresponding to each of the four possible 
eigenvalues. The dimension of each of these subspaces was worked out by 
McClellan and Park [7J, and is determined by the value of N mod 4: 





A = 1 A = -1 X = —i A = i 


N = 4m 
N — 4m + 1 
TV = 4m + 2 
N = Am + 3 


m + 1 m m m — 1 
m + 1 m m m 
m + 1 m + 1 m m 
m + 1 m + 1 m+1 m 



At this point, we might like to choose a basis of N linearly independent 
(and perhaps orthogonal) eigenvectors. But since we have entire subspaces 
full of eigenvectors, which should we choose? 
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McClellan and Park offered a simple but non-orthogonal basis in their 
original paper [JJ. Griinbaum [5] discovered another matrix that com- 
mutes with the DFT- this preserves a set of eigenvectors but breaks the 
symmetry in the eigenvalues, so we can pick out individual (and orthogo- 
nal) eigenvectors. Dickinson and Steiglitz [3] simultaneously discovered a 
different commuting matrix and used it for the same purpose. Other au- 
thors have found orthogonal bases by examining orthogonal polynomials, 
such as Kravchuk Q] or Hermite [B] polynomials. 

In this paper we take a slightly different tack- we focus on sparsity. 
The sparsity of a vector is the number of non-zero coordinates, i.e. the size 
of the support of the vector. For each dimension N, we will construct a 
basis of eigenvectors all of which are sparse- their sparsity will be within a 
factor of four of the best (sparsest) possible. If JV is a perfect square, then 
the basis is orthogonal and the eigenvectors all have sparsity 0(y/~N). It 
is possible to change to this basis efficiently by using part of a fast Fourier 
transform. 

2 Eigenspace Projections 

Let £ k be the vector subspace of C N consisting of all eigenvectors of D 
with eigenvalue i~ k . As we discussed in the introduction, all eigenvectors 
of D lie in one of So, £i, £2, or £3. We are going to examine some functions 
that project on to these four eigenspaces. Let 

F = (1/4) [I + D + D 2 + D 3 ] 

Fi = (1/4) [i + iD - D 2 - iD 3 ] 

F 2 = (1/4) [i — D + D 2 — D 3 ] 

F 3 = (1/4) [7 - iD - D 2 + iD 3 ] 

Putting it more compactly, we have F k = (1/4) Y%=o i? k iy. (Bose [2J 
and others have pointed out the utility of these functions; they can be 
viewed as factors of the characteristic polynomial.) We have constructed 
each Fk so that it projects vectors into the £k subspace. To see why this 
works, suppose we take any vector v £ (D . Then 

D(F k v) = (1/4) [Dv + i k D 2 v + i 2k D 3 v + i 3k D 4 v] 

= i" fc (l/4) [i k Dv + i 2k D 2 v + i 3k D 3 v + i 4k Iv] 
= i~ k FkV 

so FkV is an eigenvector with eigenvalue i~ k . Therefore, the image of Fk 
is contained in 

Next, suppose we choose an arbitary eigenvector v € <D N with eigen- 
value i~ k . Then 

F k {v) = [l/4)[v + i k Dv + i 2k D 2 v + i 3k D 3 v] 

1 1 / i\r 1 -h — k 1 -2k — 2fc 1 -3fc — 3k i 

= [\/A)[v + i 1 v + i 1 v + i 1 v\ 
— V 
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Therefore, the image of Fu contains £k, and hence 

Im(Fk) = £k 

The observation above provides us with a method of transforming a 
basis for into an eigenvector basis: 

Lemma 1 Suppose that {(3i, /3n} is a basis for <C N ■ Consider the set 
of vectors {Fkf3j} for k = 0, 1, 2, 3 and j = 1, N. These vectors are all 
eigenvectors of D, and it is possible to choose N of them to form a basis 
forC N . 

Proof: Fix fc. Since /3 N } span G N , then {F k fii, F k /3 N } 

span Im(Fk) = £k- Therefore, some subset of them form a basis for £u- 
Repeating this process for each k gives us a basis for all of <C N . Since the 
F k produce only eigenvectors, we are done. □ 



3 Modulated Delta Trains 

In continuous time, a delta train consists of an infinite sequence of reg- 
ularly spaced impulse functions. (This object is also called a "Dirac 
comb" .) We will examine a discrete time version of this function. Suppose 
v = (vo, Vn-i) £ C N and that N = di ■ d 2 . Then we call v a delta 
train if it is of the form 

l/\/d2 j = mod di 
else 

As with the continuous case, the discrete Fourier transform of a delta 
train is another delta train. For instance, if Dv = to, with v as above, 
then 

j = mod d 2 
else 



/ 1/V3T 
I 



We are interested in a slightly more general case- we want to be able to 
specify an offset in the non-zero entries of v and w. We can accomplish 
this effect by translation and modulation. Suppose that a and b are inte- 
gers. Then we can define the modulated delta train gd 1 (a, b) € <C N , where 
{g<ii{a,b))j is the j-ih coordinate, by: 



w / \fd~2~ j = a mod d\ 
else 



If we take the discrete Fourier transform of this vector, we find that we 
get 

m , ,» / uJ- ab Lo a3 /Va\ j = bmodd 2 

{Dg dl (a,b)h = j Q clsc 

If we factor out the uj~ ab term, we can rewrite the transformed vector as: 
Dg dl (a,b)=u~ ab g d2 {b,-a) (1) 
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So, up to a unit-length constant, the discrete Fourier transform of a mod- 
ulated delta train is another modulated delta trainQ 

Lemma 2 If we let a and b vary over the ranges < a < al\ and < b < 
di, then the set of vectors {g dl {a, b)} forms an orthogonal basis for C . 

Proof: There are di ■ di — N vectors, so orthogonality will imply 
that the vectors span and are linearly independent. To see orthogonality, 
suppose that a ^ a'. Then g dl (a,b) and g dl (a',b') have disjoint support, 
and are therefore orthogonal. On the other hand, if a = a' but 6 / 6', 
then their discrete Fourier transforms have disjoint support, i.e. 

D(g dl (a, b)) = Lo~ ab g d2 (b, -a) and D(g dl (a, b')) = u~ ab g d2 (b' , -a) 

have disjoint support, and therefore the images are orthogonal. Since D is 
a unitary transformation, that implies that the original vectors were also 
orthogonal. □ 

4 A Basis of Sparse Eigenvectors 

We are now in a position to define our basis of sparse eigenvectors. Let 
7/1 be the greatest divisor of N that is at most yN, and let 772 be the 
least divisor of N that is at least y/N. (In other words, 771 and 772 are the 
closest divisors below and above \/iV, respectively.) Note that 771-772 = N. 
We can now present our main theorem. 

Theorem 1 Take 771 , 772 as above. Then there exists a basis of eigenvec- 
tors of the discrete Fourier transform where every basis vector has at most 
2(771 +772) non-zero coordinates. 

Proof: Lemma [2] shows that the set of vectors {g vi (a,b)} forms a 
basis, where < a < 771 and < b < 772 . Lemma [T] shows that we 
can select a basis of eigenvectors from the set of all F^g vi (a,b), where 
k — 0,1, 2, 3. If we consider an arbitrary vector Fkg m (a, b) for some fixed 
k, a, b, we can write it as 

3 

F k9m {a,b) = Y, iJ " DJ WM) 

3=0 

= 9m ( a > b ) + W ab i k g V2 (b, -a) 

+i 2k gm (~ a , ~ b ) + u~ ab i ik g m (-6, a) 

Now, note that the support of g m (i.e. the number of non-zero coordinates 
in the vector) is exactly 772. Also, note that multiplying a vector by a non- 
zero constant (such as u>~ ab i k ) does not change the size of the support. 
Therefore 

\supp{F k g m {a,b))\ < \supp(g m (a,b))\ + \supp(g V2 (b, -a))\ + 
\supp(g m (-a,-b)) \ + \supp(g V2 (-b,a))\ 
= 2(771+772) 

1 We are using negative indices to simplify notation; because the index a is interpreted 
modulo d\, we could have written g d2 {b,d\ — a) for g d2 (b, —a). 
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as desired. □ 

Let us refer to the basis of sparse eigenvectors as B. We will show 
that B is nearly optimal from a sparsity perspective. We begin with some 
fundamental theorems on sparsity and the discrete Fourier transform. 

Theorem 2 Take any v g <C N . Then 

\supp(v)\ ■ \supp(Dv)\ > N (2) 

Also, suppose that d± < di are consecutive divisors of N , and that d± < 
\supp(v)\ < d2- Then 

N 

\supp(Dv)\ > — - (ofi + d 2 - \supp(v)\) (3) 
d\di 

Proof: Equation [2] was proved by Donoho and Stark A.. Equation[3] 
was proved by Meshulam [8], based on a theorem of Tao's [§].□ 

We can construct a lower bound on sparsity by applying Theorem [5] 
to eigenvectors. 

Corollary 1 If v (E is an eigenvector of the discrete Fourier trans- 
form and ni < 772 are the divisors of N defined above, then 

\supp(v)\ > (l/2)(m +772) (4) 

Proof: Whenever we apply a matrix to one of its eigenvectors (with 
non-zero eigenvalue) , the number of non-zero entries in the resulting vector 
remains constant. In particular, \supp(v)\ = \supp(Dv)\ for all eigenvec- 
tors of the DFT. Therefore, if v is an eigenvector, Equation [5] reduces 
to 

\supp(v)\ 2 > N 

and hence 

\supp(v)\ > VN (5) 

We split into two cases. Suppose N is a perfect square. Then 771 = 
T]2 — yN. Therefore, since yN = (l/2)(ni +^2), Equation [S] implies 
Equation H] 

On the other hand, suppose that N is not a perfect square. If \supp(v)\ > 
r}2, then Equation U holds immediately. Therefore, let us assume that 
\supp(v)\ < r72. Combined with Equation [5] we can conclude that 

771 < VN < \supp(v)\ < 7/2 

Since 771 and 772 are consecutive divisors of N, we can apply Equation [3] 
and conclude that 

\supp(v)\ > (771 + 772 — \supp(v)\) 

which simplifies to Equation [5J □ 

Combining Theorem [T] and Corollary [1] gives us the following result, 
which tells us that the basis B is within a factor of four of the sparsest 
possible. 

Corollary 2 Let v be any eigenvector in the basis B, and let w be any 
eigenvector of D. Then 

\supp(v)\ 
\supp(w)\ ~ 
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5 Orthogonality 



Is the basis B orthogonal? The answer depends on the dimension N. We 
begin by stating the following standard result from linear algebra. 

Lemma 3 If v and w are eigenvectors of a unitary matrix with different 
eigenvalues, then the inner product (v, w) is zero. 

This lemma implies that the £k are orthogonal subspaces. We now 
turn to the general question of orthogonality. 

Theorem 3 If N is a perfect square, then the basis B is orthogonal. 

PROOF: Because of Lemma [3] we can restrict our attention to eigen- 
vectors in £k for a fixed k; eigenvectors with different eigenvalues are au- 
tomatically orthogonal. Since TV is a perfect square, we will set n = yN 
(since 771 = 772 = V~N in this case). 

Recall, from Equation [T] that the discrete Fourier transform of a mod- 
ulated delta train is another modulated delta train: 

Dg v (a, b) = u~ ah g v (b, -a) 

Repeating this process, we find that 

D 2 g v (a,b) = g n (-a,-b) 
D 3 g v (a,b) = u~ ab g n (-b, a) 
D 4 g n (a,b) = g n (a,b) 

Recall that the Fkg v (a,b) are just weighted sums of these terms. There- 
fore, for some values of a and b, the resulting vectors Fkg v (a,b) are 
multiples each other (i.e. are collinear). For instance, Fkg v (a,b) and 
Fkg v {b, —a) are collinear for all a,b. 

More generally, if there exist j,j' such that L>^(o, b) and D J V (a , b') are 
collinear then 

Fkg v {a,b) and Fkg v {a', b') are collinear. 

Conversely, suppose that Ftg(a,b) and Fkg(a',b') are linearly indepen- 
dent. Then for any j,j', we can conclude that 

D^(o,6) and D J V (a',b') are linearly independent. 

From Lemma[5] we know that linearly independent modulated delta trains 
are orthogonal; in other words, if Fkg(a,b) and Fkg(a',b') are linearly 
independent, then 

(Dig(a,b),D j 'g(a',b')}=0 (6) 

We will return to this equation in a moment. 

Suppose we take two distinct (and hence linearly independent) vectors 
from the basis B that both lie in £k, say Ftg(a,b) and Fkg(a' ,&'), and 
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consider their inner product. We get 




(F k g(a,b),F k g(a',b')) = ( ^'^(a, 6), £ i j ' k D*' g(a' , 6') 



i'=o 



£ £ i fe(j - /) (^ 5 (a, 6), D 3 ' g(a', b') 



3=0 i'=0 

Using the assumed linear independence of our initial vectors F k g(a, b) and 
Fkg(a' ,b'), and plugging in Equation[6] we get 

= E E ikU ~ J,) ° = 

j=o i'=o 

which proves orthogonality. □ 

In addition to the perfect squares, there are several other exceptional 
values of TV for which B is orthogonal in TV dimensions. 

Theorem 4 If N — 2,3 or 8, B is an orthogonal basis. 

Proof: From Lemma [3] we know that the £k are orthogonal. From 
McClellan and Park's table of dimensions in Section [1] we know that 
if TV < 4, then dim(£fc) < 1. Therefore, the eigenvector basis is auto- 
matically orthogonal (and unique) for TV — 2 and 3. For TV = 8, the 
orthogonality can be verified by direct computation (which we will not 
repeat here).D 

Are there more of these sporadic orthogonal bases? We are not sure, 
but for every other TV < 256, there exist (o, b) and (a 1 ,b') that produce 
vectors that are neither orthogonal nor collinear. 



6 A Quick Note on Efficiency 

If we are given an arbitary vector, we may wish to represent it using our 
new sparse basis B. How efficiently can we change bases? 

Suppose we have a vector v G C N in the standard basis, and we wish 
to represent it in an arbitrary basis. Generally speaking, this involves 
multiplying the vector by an TV x TV matrix, which takes time 0(N 2 ). 
However, because all the vectors in our basis B are sparse, this corresponds 
to multiplication by a sparser matrix (unless iV is prime). For instance, if 
TV is a perfect square, the matrix multiplication only takes time 0(TV ). 

However, we can do quite a bit better than that. If we consider the 
inner product of gdi(o, 6) with v, we are taking a sum of every entry 
whose index is equivalent to a mod d\ , modulated by a complex expo- 
nential. This value is an entry of the ^-dimensional DFT applied to 
those d,2 values. If we let b vary, we can calculate all the gd 1 {a,-) inner 
products in time 0(d2logd,2) by using a FFT algorithm. There are di 
different possible values for a, so we can calculate all g& 1 (a, b) in time 
0(did,2 log 0I2) = 0(TVlogd2). We can similarly calculate the gd 2 (a,b) in 
time O(TVlogdi). In an additional O(TV) steps we can combine them to 
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form the F^ga-i {a, &)) for all a and b. Therefore, we can change basis to 
B in time 0{N log N). 

If we consider calculating the full FFT along a butterfly graph, then 
what we are really doing is halting the computation early- by reading 
off the partial results of the FFT and combining them, we calculate the 
projection onto the basis B. In the case that JV is a perfect square, the 
butterfly splits exactly into two equal pieces and we calculate half of the 
FFT. Otherwise, we find the 7/1 and 772 that get as close as possible to 
splitting the FFT in half. 
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